Prepared by Joy Kumagai and Aidin Niamir - IPBES Technical Support Unit (TSU) of Knowledge and Data
Reviewed by the Task Force on Knowledge and Data (Hanno Seebens, Rainer Krug, Gregoire Dubois, Matea Vukelić, and Xubin Pan)
For any inquires please contact tsu.data@ipbes.net
Version: 1.0 Last Updated: 19 August 2022
This technical guideline discusses how an indicator should be documented within IPBES and is intended for any experts that are involved in creating an indicator following the FAIR and CARE principles. The guideline focuses on the technical aspects of documentation.
Referenced and used within many biodiversity-related policy documents and instruments, indicators can help us to understand the current state of biodiversity at the ecosystem and species levels, threats and pressures, and conservation responses. They are incredibly important in our current science-policy interface enabling decision making from the international sphere to local responses. As defined by the Biodiversity Indicators Partnership, an indicator is “a measure based on verifiable data that conveys information about more than itself.” Indicators often take the form of final datasets resulting from a workflow. An indicator workflow is defined here as the steps performed between the input data and final indicator. Successful biodiversity indicators have a variety of attributes such as scientifically valid, updated over time, responsive to change, easily understood, and consistent as described in the document “Key Knowledge for Successful Biodiversity Indicators (Brooks & Bubb 2014).”
Clear and accessible documentation is necessary for indicators to be effective, consistent, and trustworthy. Unfortunately, indicator workflows often do not follow the FAIR (findable, accessible, interoperable, and reusable) and CARE (collective benefit, authority to control, responsibility, and ethics) principles and are not reproducible by other scientists. For example, small differences in how geospatial layers or assumptions are treated can lead to vastly different results. There needs to be transparency in the underlying calculations, to maintain consistency and ensure reuse by others. IPBES is committed to following the open science principles and increasing transparency in data management. Therefore, this guideline discusses how indicators developed within IPBES should be documented starting from the input data, through processing, to outputs to enable reuse by the community.
Throughout this guideline, we will use the indexes from the publication, Kumagai et al. 2022, as an example for documentation. The data descriptor paper published in Scientific Data describes two indexes which report on how much of six important marine habitats are within protected and conserved areas at the country and global scale
The recommendations throughout this section focus on practical steps to ensure the FAIR and CARE principles can be followed when documenting the indicator and its calculation.
It is essential to document where to find all the necessary inputs that go into the indicator workflow so others can calculate and update the indicator. Therefore, we suggest the following:
Example:
Following table 2 (shown below) within our example publication, each input dataset is listed together with its reference, the date of access and the version used to calculate the indicator. This information should be associated with the publication of the indicator as, for example, a README file.
Ensuring that others can understand the methodology and repeat the workflow of the indicator allows people to improve upon and use the indicators in their own context, greatly expanding the reach and impact of an indicator. Therefore we suggest the following:
Example:
For the habitat protection indexes, these recommendations were followed while developing the workflow. The workflow was created using open source software, R, instead of ArcGIS Pro as this ensured consistency in processing the data and there is no requirement to buy a licence to use the workflow. All scripts used to process the original datasets and to calculate the indexes can be found within the associated GitHub folder, linked respectively within the data availability and code availability sections of the publication. To enable reusability, the workflow is also structured in a user-friendly way so that the entire process can be run using one R script. Finally, figure 5 (shown below) within the publication provides an overview of the workflow to explain what each script does and how it fits into the overall process of creating the indexes.
Now that the inputs and process behind the indicator are clearly documented and explained, the final dataset (i.e. the indicator) should be findable and accessible. Additionally, in the documentation there should be information about the interpretation and limits, e.g. the scale and time frame in which the indicator can be interpreted. Therefore, we recommend the following:
Example:
These guidelines are followed within our example.
The final indicator dataset is provided in an online and open repository on Zenodo (https://doi.org/10.5281/zenodo.4694821). The final datasets are provided as CSV files (an interoperable file format), and the repository has clear metadata including a DOI, versioning, contact information, and authors’ list (see image below).
The linked publication and affiliated media reports serve to communicate the indexes to a broader audience.
The workflow scripts are also stored in the same repository with a README file (https://github.com/jkumagai96/Marine_Habitat_protection) that explains the inputs, coding environment, and the file structure to be able to repeat the workflow.
In general, the associated publication should provide all required information to reproduce the results, including a link to the workflow repository and final dataset.
Within IPBES, it is important that if any expert team develops an indicator, the associated technical support unit informs the technical support unit on knowledge and data. The technical support unit on knowledge and data can ensure that these steps are followed and all input data are accessible and stored in case the original source of the information is removed, enabling reuse by future assessments. Increasing transparency in the process of creating an indicator ensures consistency, allowing the scientific community to repeat workflows, improve upon them, and test different scenarios. Over time this process will build trust for the indicator and encourage their use to help inform policy and action.
Your feedback on this content is welcome. Let us know what other useful material you would like to see here by emailing tsu.data@ipbes.net.
References
Brooks, S. and Bubb, P. (2014) Key Knowledge for Successful Biodiversity Indicators UNEP-WCMC, Cambridge, UK 12pp. https://www.bipindicators.net/system/resources/files/000/000/410/original/901.pdf?1482313832
Kumagai, J.A., Favoretto, F., Pruckner, S. et al. (2022) Habitat Protection Indexes - new monitoring measures for the conservation of coastal and marine habitats. Sci Data 9, 203. https://doi.org/10.1038/s41597-022-01296-4